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(54) Embedding two sets of information in a digital image 



(57) A method for embedding information in a digital 
image by an imaging device that is electronically con- 
nected to a network server includes storing a digital im- 
age having first embedded information in an electronic 
memory at a network server and sending the digital im- 
age having the first embedded information from the net- 



work server to the imaging device. The method further 
includes producing second information and using the 
imaging device to embed the second information in the 
digital image having the first embedded information to 
produce a digital image having the first and second em- 
bedded information. 
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Description 

[0001 J .The present invention relates to embedding in- 
formaf ipfi in. digital images. 

[0002], .•."Images can be viewed by many different types 
of imaging devices: they can be displayed on a monitor 
such'a's a liquid crystal display (LCD), a cathode ray tub 
monitor (CRT), or an organic light emitting diode display 
(OLED). They can be printed on reflective or transparent 
receivers by printers. They can also be projected onto 
a screen using a projector to present a static image or 
motion images. For many current commercial applica- 
tions, the imaging devices are usually located at differ- 
ent sites. Modem communication tools such as the In- 
ternet allow these imaging devices to communication 
with each other or with a central location such as a net- 
work server. The digital images can be used to produce 
images on a display or print images on receivers. 
[0003] In many circumstances, the digital images are 
protected by copyright. The copyright information can 
include the owner of the image. Proper identification of 
these images needs to be provided so that the visual or 
digital images can be easily traced or authenticated. In 
some other applications, it is very desirable to attach 
information to the digital image or the visual image pro- 
duced therefrom. 

[0004] An object of the present invention is to effec- 
tively embed at least two different sets of information in 
digital images. The first set of embedded information is 
in the digital image and is stored in a network server. 
The second set of information can be produced or se- 
lected by a user. 

[0005] This object is achieved by a method for em- 
bedding information in a digital image by an imaging de- 
vice that is electronically connected to a network server, 
comprising the steps of: 

a) storing a digital image having first embedded in- 
formation in an electronic memory at a network 
server; 

b) sending the digital image having the first embed- 
ded information from the network server to the im- 
aging device; 

c) producing second information; and 

d) using the imaging device to embed the second 
information in the digital image having the first em- 
bedded information to produce a digital image hav- 
ing the first and second embedded information. 

[0006] An advantage of the present invention is that 
by embedding the first and second sets of information 
into the digital images information can be selected for 
different imaging devices, users, locations, and time so 
that each digital or visual image has specific embedded 
information. These sets of embedded information can 
be used to authenticate the digital or visual image or 
trace the history of the digital or visual image. 
[0007] FIG. 1 illustrates a system for managing, dis- 



tributing, and embedding selectable information in dig- 
ital images at different imaging devices in accordance 
with the present invention. 

[0008] This invention relates to a method for embed- 
5 ding information in a digital image by an imaging device. 
In this disclosure, the term imaging device can include 
any devices that can form a visual image, for example, 
a display device, a printer, or a projector. A display de- 
vice can be, for example, a liquid crystal device (LCD), 

10 a cathode ray tub monitor (CRT), or an organic light 
emitting diode display (OLED). The monitor can be self- 
standing or hand held. A projector projects a static im- 
age or motion images on a screen, which are exempli- 
fied by a LCD light valve or digital mirror technologies. 

*5 a printer can be an ink jet printer, a thermal resistive 
printer, a laser (electrophotographic) printer, a digital sil- 
ver halide printer, or an offset printer. The imaging de- 
vice can also include a compact disc (CD) writer for re- 
cording the digital image on a recordable CD such as a 

20 digital versatile disc (DVD) and versatile compact disc 
(VCD). In the present invention, the term imaging serv- 
ice refers to common operations that can be provided 
by the imaging devices described above. For example, 
the imaging services can include well know image 

25 processing operations, displaying, printing, scanning of 
visual images, and so forth. 

[0009] FIG. 1 illustrates an image authentication sys- 
tem 10 for providing embedding selectable information 
in digital images and producing visual images using 

30 such digital images in accordance with the present in- 
vention. The image authentication system 10 includes 
a first network server 20 having an image database 30, 
a second network server 40 having a selectable infor- 
mation database 50, a printer 70 having a processor 80 

35 for producing printed image 90, a display device 100 
having a processor 110, and an imaging kiosk 140 hav- 
ing a processor 150. The image authentication system 
10 can also include a scanner 180 having a processor 
190. The scanner 180 scans a printed image 195 and 

40 produces a digital image to be processed by the proc- 
essor 1 90 which can be subsequently stored or used to 
produce visual image. 

[001 0] The imaging devices described above produce 
visual images in response to the request of the users. 

45 The visual image can be produced by displaying on a 
monitor or by printing on a reflective or transparent re- 
ceiversuch as paper or synthetic material. The usercan 
select at least one property for producing such visual 
images. For the printer 70, for example, the selectable 

50 image properties includes image size, the number of the 
visual images, the properties of the imaging device in- 
cluding the printer type, the printing resolution, the prop- 
erties of the colorants, the receiver type, and the print 
finish. The selectable properties can also include imag- 

55 ing processing operations for enhancing the features or 
the quality of the visual images. The image processing 
operations include adjusting tone scale, adjusting color 
balance, adjusting color saturation, reducing red-eye, 
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sharpening of softening the image, reducing blur, crop- 
ping, zooming, inserting the digital image into an image 
template,, adding text or annotation into the image, and 
retouching' the image. 

[0011] • Tne imaging devices can be located adjacent 
to each other/or installed at different locations such as 
shopping malls, airports or train stations, movie cinema, 
entertainment parks, or supermarkets so that users can 
easily access to imaging service at their convenience. 
The imaging devices are in electronic communication 
with the first network server 30 and the second network 
server 40. The first network server 30 is installed at a 
first service provider that provides the digital images that 
are used at various imaging devices. The second net- 
work server 40 having the selectable information data- 
base 50 is installed at a second service provider. The 
second service provider stores and manages the se- 
lectable information, distributes information to imaging 
devices so that the selectable information can be em- 
bedded in digital and visual images at the imaging de- 
vices. The second network server sends selectable in- 
formation to each imaging device and such selectable 
information can be specific to each imaging service. In 
some applications, the first and the second service pro- 
viders can be the same. In such a case, the selectable 
information database 50 and the image data base 30 
can be located on the same network server 20 or 40. In 
the other applications, the first and the second service 
providers are different. Proper payment structure needs 
to be set up. The first service provider uses the service 
from the second service provided for selecting selecta- 
ble information and embedding such information in dig- 
ital images at the imaging devices. The first service pro- 
vider can pay a fixed fee to the second service provider 
for a fixed period of time, or pay for embedding informa- 
tion for each imaging service. The price information can 
also be included as part of the image access information 
as described below. Payment can be made by a user 
providing a payment identifier which specifies an ac- 
count, such as a credit card, from which payment is to 
be made. A user enters a payment identifier and makes 
payment for the services that have been provided. 
[0012] A communication channel 200 electronically 
connects the network servers 20 and 40, the printer 70, 
the display device 100, the imaging kiosk 140, and the 
scanner 180 for providing communication between 
these devices. The communication channel 200 can ex- 
ist in different physical and software forms. For example, 
the communication channel 200 can include an Internet 
or a local network. They can be built with an optical-fiber 
cable, a co-axial cable, telephone line, satellite trans- 
mission system, and wireless radio-wave transmission. 
The communication over the communication channel 
200 can use different protocols such as the World Wide 
Web, and file transfer protocol (FTP). 
[0013] For certain applications, it is beneficial to pro- 
vide imaging service from a central location, which may 
be in the vicinity of the first and second network servers 



20 and 40. For example, an offset printer can be provid- 
ed at a central location where a large number of printed 
images can be made. The requests can be entered by 
the users from home or the distributed imaging devices. 
5 The user requests are sent to the first and the second 
network servers 20 and 40. The network servers then 
allocate the printing jobs to the offset printer. The printed 
images 90 can be mailed to the user designated ad- 
dresses. The printed images can be photographs, tick- 
to ets, postal stamps, or product packages. 

[0014] In all above situations, the present invention 
provides embedding of selectable information in the dig- 
ital images. Such embedded information can include 
copyright information. Such information can also include 
is how the image is produced (capture or creation date, 
the type of capturing device used), the event that the 
image is associated with (graduation, birthday party and 
so forth.), and image calibration parameters (tone and 
color calibration). As described below, the embedded in- 
20 formation is selectable specific to how and when the vis- 
ual image is produced. The information embedded can 
specify the user, the imaging device, the location, and 
the time related to producing the digital image, or the 
formation of the visual image at the imaging devices. 
25 The embedding of the selectable information can be im- 
plemented at the imaging devices or by a different com- 
puter such as the Network server 20. 
[0015] In accordance with the present invention, the 
digital images stored on the image database 30 can al- 
so ready include previously embedded information. The 
previously embedded information can include, for ex- 
ample, image-access information or authorization infor- 
mation . The authorization information is used for permit- 
ting a user to produce a visual image at an imaging de- 
35 vice. The authorization information permits the use with 
and without charge of the digital image for forming a vis- 
ual image on an imaging device. The authorization in- 
formation may also prohibit the use of the digital image 
for certain applications. The authorization information 
40 can also authorize the operation of a device. For exam- 
ple, the authorization information can authorize the cop- 
ying operation of a compact disc writer to copy the digital 
image files from a memory to a recordable disc. The au- 
thorization information can authorize the use of a soft- 
45 ware stored in an imaging device or a computer. 

[0016] The image-access information can also in- 
clude information about the receiver of payment for the 
imaging services using these digital images. An exam- 
ple of the receiver of payment is the first service provider 
50 for the imaging services. The image-access information 
can also include the web address (URL), or e-mail ad- 
dress of the receiver of the payment such as the service 
provider. In the present invention, an image service can 
include capturing, processing, printing, copying, scan- 
55 ning, and reproducing of images in an electronic device 
or to or from a receiver. 

[0017] The image-access information can also in- 
clude features for forming the visual image to be pro- 
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duced on a receiver or a monitor. For example, the im- 
aging programs can include adjusting tone scale, ad- 
justing color balance, adjusting color saturation, reduc- 
ing red-eye, sharpening of softening the image, reduc- 
ing blur, cropping, zooming, inserting the digital image 
into an image template, adding text into the image, and 
retouching the image. The image-access information 
can also include image size and the number of the visual 
images to be produced, and properties of the imaging 
device such as the printer or the scanner type, the print- 
ing resolution, the properties of the colorants, the receiv- 
er type, and the type of print finish. These device prop- 
erties often determine the quality of the visual images 
as well as the cost to the service provider. The payment 
for the imaging service can depend on these above de- 
scribed features and properties. 

[0018] For providing imaging services to the user at 
an imaging device such as the printer 70, the display 
device 1 00, imaging kiosk 140 or a scanner 1 80, the im- 
aging device receives digital images from the image da- 
tabase 30. Image-access information is embedded in 
the digital image. After the digital images are received, 
the embedded image-access information is extracted 
from the digital images, as described below, in the proc- 
essors 70, 1 00, 1 40, 1 80 that are located within or near 
each of the imaging devices. 

[001 9] If the digital image received by the imaging de- 
vice is previously embedded with information such as 
the image access information, the previously embedded 
information can be either removed or kept in the digital 
image after the previously embedded information is ex- 
tracted in accordance with the present invention. The 
previously embedded information in the digital image 
can be removed. Since such removal of previously em- 
bedded information can be made lossless, the digital im- 
age is thus free of the noise related to the previously 
embedded information. The noise level in the digital im- 
age can be reduced. The previously embedded informa- 
tion in the digital image can also be kept and the addi- 
tional selectable information is further embedded at the 
imaging devices. The resulted digital image has then 
both the previously embedded information and the fur- 
ther embedded selectable information at the imaging 
device, which can subsequently used to produce visual 
images on the display or a printer. 
[0020] Composite images can be formed at the imag- 
ing devices such as the printer 70, the imaging kiosk 
1 40 and display device 1 00. The digital image sent from 
the image database 30 to the imaging devices 
(70,100,140) can be used as an image object that is a 
part of the composite image. For example, the digital 
image may include the face image of a person. It can 
be combined with a frame or theme image or annotation 
to form a composite image. The image object from the 
digital image sent from the image database 30 can con- 
tain previously embedded information as described 
above. The new composite image can contain selecta- 
ble information embedded for the particular imaging 



service at the imaging device. The composite images 
can be used to produce visual images. This way, the 
history of the visual images and digital images can be 
traced be by detecting the existence of the previously 
5 embedded information and the newly embedded se- 
lectable information. As described above, the previously 
stored information can be related to copyright, image ac- 
cess, and capture information, (calibration information), 
which can be generally related to the service provider 

10 or can be specific to the use of each digital image. The 
embedded selectable information specifies each partic- 
ular imaging service, for example, the information about 
the user, the identity of the imaging device, the location 
and the time when the digital image is embedded with 

is the selectable information or when a visual image is pro- 
duced from the digital image. The selectable information 
is selected for each imaging service. The selection can 
be made by the service provider, or the user, and can 
be automated by the processors 70, 100, 140, 180. 

20 [0021] The embedding of the selected selectable in- 
formation will now be described. A digital image l(x,y) is 
stored in image database 30. It will be understood that 
the digital image can include previously embedded in- 
formation so this process produces a digital image hav- 

25 ing first and second embedded information. The select- 
ed selectable information is embedded into the digital 
image l(x,y) by modifying the pixel values in l(x ; y) to pro- 
duce a modified digital image l'(x,y) having the embed- 
ded selected selectable information. The selected se- 

30 lectable information can be encrypted before embed- 
ding. 

[0022] The embedding of the selected selectable in- 
formation starts with producing a message image M(x, 
y). The message image M(x,y) can include alphanumer- 

35 ic or graphic information associated with the selected 
selectable information. A carrier image C(x,y) is also 
generated with a uniform or flat Fourier amplitude spec- 
trum and a phase spectrum generated using a random 
number generator for each phase component. The ran- 

*o dom phase is required to distribute the message image 
to make the message invisible and unreadable to unaid- 
ed human eyes. 

[0023] The message image M(x,y) and the carrier im- 
age C(x,y) are then convolved to form a distributed sig- 
45 nal S(x,y). 

S(x,y)=M(x 1 yYC(x ) y) (1) 

50 in eqn. (1 ), the symbol "*" denotes the mathematical op- 
eration of convolution, for example, a circular convolu- 
tion. 

[0024] The distributed signal S(x,y) is next added to 
the digital image l(x,y) to produce a modified digital im- 
55 age l'(x,y) carrying the embedded information: 

l'{x t y)= l(x,y) + aS(x,y) (2) 
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The distributed signal S(x,y) is scaled by a scale factor 
a to maximize the signal of the embedded selected se- 
lectable information while ensuring the invisibility of the 
distributed message in the modified digital image l'(x,y). 
That is, the value of a can also be chosen for optimizing 
image quality. 

[0025] The reliability of extracting the embedded in- 
formation can be improved by superimposing a plurality 
of blocks of distributed signals S(x t y) to each color plane 
of the processed digital image l(x,y). Each of the blocks 
can be, for example, 64 x 64, 128 x 128, or 256 x 256 
pixels in size. For a 300 dpi or 600 dpi printer, these 
blocks represent a small area (smaller than 1 square 
inch) on the printed identification document. The redun- 
dancy in the distributed signals S(x,y) increases the sig- 
nal-to-noise ratio in the extraction of embedded infor- 
mation from modified digital image T(x,y). In the present 
invention, the term extracting is used to refer the action 
of retrieving information from an image or visual textual 
information. 

[0026] The modified digital image l'(x,y) is transferred 
via the communication channel 200 to an imaging de- 
vice such as the printer 70. The selected selectable in- 
formation is extracted from the modified digital image P 
(x.y)inthe processors 70, 100, 150, 180. The extraction 
of the embedded selected selectable information from 
the modified image l'(x,y) involves cross -correlating l'(x, 
y) with the carrier image C(x,y) to obtain the message 
image M(x,y). If the selected selectable information is 
encrypted, the selected selectable information is subse- 
quently decrypted from the extracted information by the 
processors 70 : 100, 150, 180 at the imaging devices. 
[0027] Before the visual image is finalized, the user 
can preview the improved (that is processed or modi- 
fied) image on a monitor. The user can order the final 
visual image such as the number or the size of the visual 
images according to her satisfaction with the preview 
image. After the order is made for the imaging service, 
the proper charge is debited from the user's account to 
the receiver of the payment based on the selected im- 
age property. The visual image having the image prop- 
erty is subsequently produced on or by the imaging de- 
vice. 

[0028] The selectable information can be embedded 
intothe digital image atthefirst network server20 before 
the digital image having the embedded selectable infor- 
mation is sent to an imaging device. Or alternatively, an 
imaging device receives a digital image from the first 
network server 20 and selected selectable information 
from the second network server 40. The selected se- 
lectable information can be encrypted before being 
transferred over the communication channel 200. The 
selectable information is subsequently embedded into 
the digital image at that imaging device. 
[0029] The digital image having the embedded select- 
ed selectable information can be written on to a CD by 
a CD writer or stored in a memory. The digital image 
having the embedded selected selectable information 



can be viewed on a display or printed by a printer to 
produce a visual image. Preferably, the embedded se- 
lected selectable information is invisible to unaided hu- 
man eyes in such a visual image. 
5 [0030] The embedded selected selectable informa- 
tion in the digital image or the visual image herein pro- 
duced can subsequently be extracted from the digital 
image l'(x,y) or after printing scanning then extract. The 
extracted selectable information can be used for identi- 
fy fication and authentication of the digital or visual images 
or for tracking or tracing illicit use. 
[0031] The data embedding technique disclosed in 
US-A-5, 636,292 is also compatible with the present in- 
vention. The disclosures in these references are hereby 
15 incorporated in regard to embedding and extracting se- 
lected selectable information in the digital images stored 
in the image database 30. 

[0032] A computer program product may include one 
or more storage medium having instructions for causing 

20 a computer to perform the method of the present inven- 
tion. For example, the computer program product can 
include magnetic storage media such as magnetic disk 
(such as a floppy disk) or magnetic tape; optical storage 
media such as optical disk, optical tape, or machine 

25 readable bar code; solid-state electronic storage devic- 
es such as random access memory (RAM), or read-only 
memory (ROM); or any other physical device or media 
employed to store a computer program having instruc- 
tions for practicing a method according to the present 

30 invention. The computer program can be stored on a 
storage medium in or connected to the imaging device 
or in one or more of the network servers. 
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a digital image by an imaging device, comprising 
the steps of: 

< i. ■ ■ 

V;v*a). storing a digital image and selectable infor- 
mation in an electronic memory at a network 
. ,. ; server; 

b) selecting information from the selectable in- 
formation; 

c) . sending the digital image and the selected 
selectable information from the network server 
to the imaging device; and 

d) using the imaging device to embed the se- 
lected selectable information in the digital im- 
age. 

The method of claim 1 wherein a user selects the 
selectable information. 

The method of claim 1 wherein each network server 
is connected to a plurality of imaging devices and 
the network server sends different selectable infor- 
mation to each imaging device. 

The method of claim 1 wherein the step of embed- 
ding includes modifying a predetermined number of 
pixel values of pixels in the digital image in response 
to the selected selectable information. 

The method of claim 1 wherein the embedded se- 
lected selectable information in the digital image 
can subsequently be extracted. 
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The method of claim 1 wherein the digital image 
having the embedded selected selectable informa- 
tion is produced by a CD writer. 35 

A method for embedding selectable information in 
a digital image by an imaging device that is elec- 
tronically connected to a network server, compris- 
ing the steps of: 40 



a) storing a digital image and selectable infor- 
mation in an electronic memory at a network 
server; 

b) selecting information from the selectable in- 
formation; 

c) producing a plurality of selectable informa- 
tion which specifies the user, the imaging de- 
vice, the location, and the time related to pro- 
ducing the digital image; 

d) selecting the selectable information; and 

e) using the imaging device to embed the se- 
lected selectable information in the digital im- 
age. 



45 



50 



a) storing a digital image and selectable infor- 
mation in an electronic memory at a network 
server; 

b) selecting information from the selectable in- 
formation; 

c) selecting the selectable information; 

d) sending the digital image and the selected 
selectable information to the imaging device; 

e) using the imaging device to embed the se- 
lected selectable information in the digital im- 
age; and 

f) making payment for embedding the selecta- 
ble information. 

A method for embedding information in a digital im- 
age by an imaging device that is electronically con- 
nected to a network server, comprising the steps of: 

a) storing a digital image having first embedded 
information in an electronic memory at a net- 
work server; 

b) sending the digital image having the first em- 
bedded information from the network server to 
the imaging device; 

c) producing second information; and 

d) using the imaging device to embed the sec- 
ond information in the digital image having the 
first embedded information to produce a digital 
image having the first and second embedded 
information. 



1 0. A method for embedding information in a digital im- 
age by an imaging device that is electronically con- 
nected to a network server, comprising the steps of: 

a) storing a digital image having first embedded 
information in an electronic memory at a net- 
work server; 

b) sendingthe digital image having the first em- 
bedded information from the network server to 
the imaging device; 

c) removing the first embedded information 
from the digital image to produce a digital image 
without the first embedded information; 

d) producing second information; and 

e) using the imaging device to embed the sec- 
ond information in the digital image without the 
first embedded information to produce a digital 
image having the second embedded informa- 
tion. 
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A method for embedding selectable information in 
a digital image by an imaging device, comprising 
the steps of: 
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